[engine] rm python graphmaker; create dot formatted display #4295

baroquebobcat · 2017-03-01T00:41:55Z

Problem

Printing rule graphs has not been converted. Also, generating subgraphs.

Solution

(describe the modifications you've made.)

Result

(describe how your changes affect the end-user behavior of the system. this section is
optional, and should generally be summarized in the title of the pull request.)

stuhood

Thanks Nick.

stuhood · 2017-03-01T17:18:55Z

src/rust/engine/src/lib.rs

+  product_type: TypeConstraint
+) -> RuleGraph {
+  let graph_maker = GraphMaker::new(&raw.scheduler.tasks,
+                                    RootSubjectTypes { subject_types: vec![subject_type.clone()] });


This struct doesn't appear to be carrying its weight. Could either make it a tuple struct so that you can construct it more easily (RootSubjectTypes(vec![subject_type.clone())) or just remove it, probably.

Yeah. I added it thinking it might accrete more behavior, but then it didn't. xxing

stuhood · 2017-03-01T17:19:42Z

src/rust/engine/src/lib.rs

+  let file = try!(OpenOptions::new().append(true).open(path));
+  let mut f = BufWriter::new(file);
+
+  try!(write!(&mut f, "{}\n", format!("{}", graph)));


We've mostly switched to ? rather than try!... will make a pass over the old code at some point, but might as well use ? in new code.

updating to it.

stuhood · 2017-03-01T17:20:24Z

src/rust/engine/src/lib.rs

+}
+
+fn write_to_file(path: &Path, graph: &RuleGraph) -> io::Result<()> {
+  let file = try!(OpenOptions::new().append(true).open(path));


Append doesn't seem like a good default... I don't want a graph appended to an existing file, since dot isn't appendable afaik?

Makes sense. This was cargo culted from one of the other file creating fns. -- trace in graph.rs

stuhood · 2017-03-01T17:22:27Z

src/rust/engine/src/rule_graph.rs

+        unfulfillable_rules: full_unfulfillable_rules,
+      }
+    }
+    let beginning_root = beginning_root_opt.unwrap();


A cleaner way to work with an Option while still returning early might be:

let beginning_root = if let Some(beginning_root) = self.gen_root_entry(subject_type, product_type) { beginning_root } else { return ... };

yeah, my python / java is showing a bit there.

stuhood · 2017-03-01T17:26:08Z

src/rust/engine/src/rule_graph.rs

+    full_dependency_edges = constructed_graph.rule_dependency_edges.clone();
+    full_unfulfillable_rules = constructed_graph.unfulfillable_rules.clone();
+
+    self.add_unreachable_rule_diagnostics(&full_dependency_edges, &mut full_unfulfillable_rules);


It certainly seems like you should be able to pass these in without cloning them... is the issue just that constructed_graph isn't mutable? let mut constructed_graph = ...

Yep. I started to see that when I extracted the method. Thanks for pointing it out.

stuhood · 2017-03-01T17:27:31Z

src/rust/engine/src/rule_graph.rs

+      unfulfillable_rules: full_unfulfillable_rules
+    };
+
+    self._remove_unfulfillable_rules_and_dependents(unfinished_graph)


Another reminder that this pattern of methods that take a value, mutate it, and then return the same value... feels weird to me. I would expect a method that mutates something and then gives it back to take a &mut to the thing.

So instead of taking ownership, modifying and passing back ownership, just doing an &mut is more idiomatic?

Yes, I believe so. I'm not sure whether LLVM or rustc will sneakily turn the call into a reference for you, but afaik: moving values in is ... literally moving (ie memcpying) them. The graph is not a large struct (it only has Vecs in it, and they are always 3 words long?) but I think the idea is that when you can pass by reference, you should.

stuhood · 2017-03-01T17:30:07Z

src/rust/engine/src/rule_graph.rs

@@ -602,6 +690,54 @@ impl RuleGraph {
  }
 }

+impl fmt::Display for RuleGraph {


This seems like an abuse of Display. I wouldn't accidentally want to drop this in a log message, for example.

Yeah. I agree. I was planning on taking this out, but with the support rotation stuff---I wanted to start the CI runs before fixing it.

stuhood · 2017-03-01T17:31:22Z

src/rust/engine/src/tasks.rs

@@ -71,7 +71,11 @@ impl Tasks {
  }

  pub fn all_product_types(&self) -> Vec<TypeConstraint> {
-    self.all_rules().iter().map(|t| t.product).collect()
+    let mut product_types: Vec<_> = self.all_rules().iter().map(|t| t.product).collect();
+    // NB sorted by id so that dedup will consolidate runs of duplicates.


Any reason not to just collect into a hashset instead?

I think that should work.

stuhood

Thanks!

stuhood · 2017-03-02T21:49:13Z

src/rust/engine/src/lib.rs

-  try!(write!(&mut f, "{}\n", format!("{}", graph)));
+  let file = File::create(path)?;
+  let mut f = io::BufWriter::new(file);
+  graph.visualize(&mut f)?;


Rather than:

graph.visualize(&mut f)?; Ok(())

... you can just return the result of visualize:

graph.visualize(&mut f)

…ld#4295) ### Problem Printing rule graphs has not been converted to Rust. This can make it hard to visualize what the rule sets look like. Also, the Python rule graph construction is effectively dead code already. ### Solution Port rule graph visualization and visualization based testing to the rust side of the engine implementation. ### Result Now we're setup to use the RuleGraph from the Rust side for functions as described in pantsbuild#4303 and pantsbuild#4304.

[engine] rm python graphmaker; create dot formatted display

d51e93a

stuhood reviewed Mar 1, 2017

View reviewed changes

Stu's comments

8859afd

stuhood approved these changes Mar 2, 2017

View reviewed changes

baroquebobcat added 3 commits March 2, 2017 15:00

just return the result of visualize directly

f103b04

Merge branch 'master' into nhoward/rm_graph_maker

df5f362

Merge branch 'master' into nhoward/rm_graph_maker

8eaec05

baroquebobcat merged commit 45507e2 into pantsbuild:master Mar 7, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[engine] rm python graphmaker; create dot formatted display #4295

[engine] rm python graphmaker; create dot formatted display #4295

baroquebobcat commented Mar 1, 2017

stuhood left a comment

stuhood Mar 1, 2017

baroquebobcat Mar 2, 2017

stuhood Mar 1, 2017

baroquebobcat Mar 2, 2017

stuhood Mar 1, 2017

baroquebobcat Mar 2, 2017 •

edited

Loading

stuhood Mar 1, 2017

baroquebobcat Mar 2, 2017

stuhood Mar 1, 2017

baroquebobcat Mar 2, 2017

stuhood Mar 1, 2017

baroquebobcat Mar 2, 2017

stuhood Mar 2, 2017

stuhood Mar 1, 2017

baroquebobcat Mar 2, 2017

stuhood Mar 1, 2017

baroquebobcat Mar 2, 2017

stuhood left a comment

stuhood Mar 2, 2017

[engine] rm python graphmaker; create dot formatted display #4295

[engine] rm python graphmaker; create dot formatted display #4295

Conversation

baroquebobcat commented Mar 1, 2017

Problem

Solution

Result

stuhood left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

baroquebobcat Mar 2, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stuhood left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

baroquebobcat Mar 2, 2017 •

edited

Loading